Hypothesis Testing, as such an important statistical technique applied widely in A/B testing for various business cases, has been relatively confusing to many people at the same time. This article aims to summarize the concept of a few key elements of hypothesis testing as well as how they impact the test results. The story starts from hypothesis. When we want to know any characteristics about a population like the form of distribution, the parameter of interest(mean, variance etc.), we make an assumption about it, which is called the hypothesis of population. Then we pull samples from population, and test whether the sample results make sense given the assumption. For example, your manager somehow knew that the mean of the click-through-rate per user from company's website across the user base is 0.06(mean of CTR of population), while you doubt that and believe the CTR should be higher.